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Using quantitative assessment, Montana State 
University—Billings substantially improved 
and expanded its developmental education 
program and learning center during the past 
five years. Student-centered questions drove 
the research efforts. By gathering, analyzing 
and sharing hard data, the department 
identified unmet student needs, discovered 
trends, dispelled myths about developmental 
education students, and added validity to 
the program while gaining administrative 
support. 


Five years ago, Montana State University (MSU)-Billings reorganized 
its developmental education program by combining separate math and 
writing tutoring centers into one comprehensive learning center and 
creating an administrative team to oversee the learning center and the 
instruction of three developmental education courses: English 100, Math 
101 Introductory Algebra, and Math 105 Algebra for College Students. 
Under the new structure, the program thrived. The total number of 
developmental education sections offered increased 40 percent, from 40 
sections in Fall 2001 to 56 sections in Fall 2005. In addition to the 
main learning center, tutoring services expanded to include a learning 
center on the satellite College of Technology campus in Summer 2004. 
A partnership with the local public school district resulted in an adult 
education classroom opening at the College of Technology in Fall 2004. 
Math 085 Math Fundamentals was created and offered for the first time 
Fall 2005. Fall 2006 marks the beginning of a developmental education 
reading course, and an English 085 is being developed for Fall 2007. 

In addition to expanding course offerings, the learning center 
expanded its services from assistance with math, writing, and reading 
to include tutoring in anatomy and physiology, business, psychology, 
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foreign languages, accounting, chemistry, physics and other specialty 
areas for specific majors and programs. Student visits to the main campus 
learning center average 600 a day. At the College of Technology, student 
visits to the new tutoring center reached 100 a day after just one year in 
existence. Both numbers are remarkable considering MSU—Billings’ total 
enrollment hovers around 4,000, with approximately 1,000 students at 
the College of Technology. 

What lies behind the growth and success of MSU-Billings’ 
developmental education program and learning center? Quantitative 
assessment. Facts gleaned from this assessment form the foundation of 
every decision. Gathering and analyzing data enables the administrative 
team to identify unmet student needs, discover trends, dispel “myths” 
about developmental students, and add validity to the program to gain 
administration support. 

Using Quantitative Measures to Improve Services 

In Lewis Carroll’s Alice in Wonderland, Alice meets the Cheshire Cat 
in the woods and seeks direction. After finding out Alice does not care 
where she goes, the Cat replies, “Then it doesn’t matter which way you 
walk.” But, retorts Alice, “I want to get somewhere.” “Oh, you’re sure 
to do that,” smiles the Cat, “if you only walk long enough” (Carroll, 
1957, p. 57). Developmental education programs operate on too few 
resources and have too little time with students to wander through a 
forest, hoping to get somewhere. As Flendriksen, Yang, Love and Hall 
(2005) wrote, “...at a time of shrinking budgets, growing enrollments, 
and changing student demographics, we can no longer assume that we 
are meeting students’ needs” (p. 57). Using quantitative measures in 
decision-making not only establishes an end goal, to provide students 
with the best program possible to prepare them for academic success, but 
also helps administrators identify which paths to take to achieve those 
goals, even creating signposts when forks in the road appear. 

Without actual figures as a foundation, decisions are nothing more 
than educated guesses at best and ill-conceived whims at worst. Yet, a 
survey by Boylan, Bliss, and Bonham (1997) found that only 25 percent 
of four-year universities engage in a systematic assessment of their 
programs. When the developmental education program at MSU-Billings 
reorganized five years ago, the department began gathering data to eval¬ 
uate the program and make needed changes, resulting in its growth. But, 
how does one determine what data to gather? 
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Gathering Data 

Deciding what data to gather, like Alice deciding which path to take, 
begins with knowing the desired outcome. Unlike Carroll’s Alice, we 
know our desired destination. We want to ensure that students receive an 
opportunity to succeed by having their needs met through developmental 
education courses. Placing the student, instead of the program itself, at 
the center of the assessment keeps the result focused where it should 
be: on providing the best assistance to students. Too often research 
concentrates on the program or tutoring center instead of the student 
(Simpson, 2002). Asking questions such as, “Are students prepared for 
future academic success?” and, “Are the students’ needs being met?” 
places the student at the core of the research. 

Using student-centered questions to guide the assessment moves data 
collection beyond the traditional measures. Historically, studies have 
measured course completion rates, college retention rates, and grade 
point averages (Boylan & Saxon, n.d.). However, those measures only 
capture part of a larger picture. Data collection needs to take a longitu¬ 
dinal look at the student and ask: What can we do to meet student needs 
before they enter the program, what can we do to help students succeed 
while they are in the program, and how can we tell if the students have 
succeeded after they have left the program? 

To determine the answer to the first two questions, MSU-Billings 
relies heavily on the COMPASS placement exam to gather scores in 
math, writing, and reading during new student orientations, which 
begin in the spring and continue throughout the summer. In addition 
to placement scores, we collect information about students’ academic 
standing before, during and after their developmental education 
experience. To determine if student needs are being met and if students 
are being prepared while enrolled in the developmental education courses, 
we collect diagnostic exam scores, gain scores (improvement from pre- 
developmental education to post-developmental education), midterm 
grades and final grades. Collecting course withdrawal information and 
the effects of retention efforts, such as contacting all students who did 
not attend the first two weeks of class, sending letters to students failing 
or in danger of failing at midterm, and contacting students who can no 
longer possibly pass a class prior to the withdrawal deadline, provides 
information about student retention. 

A student’s performance after completing developmental education 
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courses measures a program’s true effectiveness. Post-developmental 
education course grades; future academic standing, such as probation, 
suspension, and honor roll; graduation rates; and student honors and 
awards help answer the final question: Are students succeeding after 
they leave the program? At many universities and colleges, a statistician 
or student database already holds much of the information needed for 
analysis. If such resources are not available, databases and spreadsheets 
can be created using standard software packages, such as Excel®. 

Analyzing Data 

“Number-crunching” remains the first step in analyzing data after it 
is collected. Calculating percentages, such as the percentage of students 
withdrawing or receiving Ds and Fs, reveals what happened in a particular 
course during a particular term. However, data analysis needs to address 
the “why” as well. As Simpson (2002) states, the “why” question digs at 
the reason, the underlying cause that can be changed to improve student 
success. For example, examining the percentage of students not success¬ 
fully completing a course leads to the question, “Why are these students 
not succeeding?” Administrators, educators, and researchers can then 
begin to look for information that can answer the deeper question. For 
example, did those students not succeed because of lack of ability or 
poor attendance? What were those students’ placement scores? What 
was their attendance behavior: Did they attend class regularly; did they 
stop attending after midterm? One analysis leads to another, creating a 
pathway that can lead to changes. 

Enhancing Program and Student Success 

Collecting data and endeavoring to answer the “why” questions 
while analyzing the data improves the program by identifying unmet 
student needs and justifying program changes. At MSU-Billings, the 
combination of questions asked and data gathered led to a close scrutiny 
of placement scores. Were the placement scores appropriate for our 
population? Analyzing pre- and post-developmental education place¬ 
ment scores and course completion data showed that certain portions 
of students were not successfully completing the courses. As a result, we 
adjusted the placement scores and created additional courses. 

Specifically, in both math and English, the lowest-scoring students were 
not passing the developmental English 100 or Math 101. We compared 
placement exam scores to course grades and found a correlation existed. 
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Not surprisingly, the higher the placement score, the better the grade. 
However, we found a certain placement score below which virtually 
no students passed the courses. Also, to better serve the students, we 
created a partnership with the school district and opened a satellite 
adult education classroom on the College of Technology campus. 
Advisors directed students scoring below a 23 on the writing portion 
of the COMPASS placement exam to adult education to improve 
their skills before attempting English 100. On the math side, advisors 
directed students scoring less than a 35 on the pre-algebra portion of 
the COMPASS exam to adult education. Thus, the very low-performing 
students could increase their skills for free with adult education before 
attempting the developmental education courses and improve their 
capacity for success in those courses. In addition, we created a Math 085 
course to instruct students scoring between 36 and 100 in pre-algebra 
or 0 to 15 in algebra to better prepare them for success in the Math 101 
course. We also increased the placement score to enter Math 105 from 
a 21 to a 27 COMPASS algebra score, effectively placing more students 
in Math 101. Students scoring between 21 and 27 proved more likely 
to fail Math 105 without completing Math 101 first. They were not 
yet prepared for the coursework. After adjusting the math COMPASS 
entrance scores, the standard deviation of improvement in the math 
classes decreased, meaning more students were experiencing the same 
amount of improvement. 

An examination of student gain scores revealed the need for a lower 
level English course, English 085. The standard deviation was much 
greater for English 100 than the revised math courses. Students in English 
100 were not consistently improving their performance. Why? Without 
English 085, students scoring a 23 on the writing portion enrolled in the 
same English 100 classes as students scoring an 86 (an 87 is required 
for entrance to a non-developmental education English 150 course). 
Instruction geared to preparing students for English 150 left many 
lower-performing students behind. Struggling students needed more 
basic grammar and sentence structure instruction, while the majority of 
the class engaged in writing essays in specific formats. Work began on 
developing an English 085 course to meet those students’ needs. 

An analysis of the completion rates and retention efforts revealed the 
need to direct extra attention to one particular group of students: those 
attempting multiple times to pass a course. Students enrolled in courses 
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for the second or third time seemed to be more resistant to intervention 
efforts than were those attempting the course for the first time. For 
example, a lower percentage of multiple-attempt students responded to 
midterm warnings than those enrolled in the class for the first time. The 
multiple-attempt students were more likely to receive a failing grade even 
though they had been through the course before. The data also revealed 
that early intervention worked best. The earliest interventions succeeded 
more often than subsequent efforts. Combining those two pieces of 
information led us to add personal phone calls and “hall visits” with 
multiple-attempt students early in the semester instead of just sending 
the standard letters. 

Another example of using quantitative assessment to identify and meet 
student needs started with a student-centered “how” question. How were 
students with low reading scores performing? The university had collected 
COMPASS reading scores but had not analyzed the data. Statistical 
calculations revealed a positive correlation between reading score and 
GPA. In addition, by comparing reading scores to retention data, such as 
persistence semester to semester, GPA, and academic standing, we found 
students who scored below an 80 on the COMPASS reading exam were 
significantly more likely to experience academic failure after the first 
year than students who scored above an 80. Because we had the objective 
data, the university’s administration approved the implementation of a 
reading course designed to help students improve their college reading 
ability and performance. We had more than a “hunch” and more than 
instructors’ anecdotal comments about their students’ poor reading 
abilities to justify the expense of a new course. 

Sharing Data 

After gathering and analyzing data, the last step in using quantitative 
assessment to enhance program and student success is to share the data. 
Share the information with instructors, tutors, and administrators. Help 
them see the importance and success of developmental education. Dispel 
myths by sharing the information with other academic departments, 
especially those who matriculate with the program. Build support by 
sharing the information with other support services: advising, housing, 
campus security, financial aid, and other student programs. The advising 
staff, for example, appreciates the tracking of multiple-attempt students. 
Housing and financial aid personnel display particular interest in 
students’ attendance patterns. 
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Furthermore, the importance of using quantitative assessment to 
gain administrative support cannot be overstated. When the battle 
begins to rage between departments for funding, when universities 
stretch to increase retention efforts, when dollars and jobs are on the 
line, departments built on a bedrock of quantitative assessment can 
withstand the buffeting squalls and torrents. When certain university 
factions declared MSU-Billings’ developmental education department 
was not adequately preparing students for subsequent classes, we dashed 
that misconception by rolling out the numbers: 52 percent of students 
with a previous developmental education math course achieved a C or 
better in subsequent math courses compared to 48 percent of students 
in the same courses who had not taken a developmental education math 
course. In addition, 55 percent of students with a previous developmental 
education English course achieved a C or better in the non-developmental 
education college composition class compared to 45 percent of students 
without a developmental education English course. 

Elard facts dispelled other myths. Opponents said developmental 
education students could succeed at the developmental education level, 
but they could not survive through graduation. Again, because we had 
been gathering and analyzing data, we could show that 18 percent of the 
bachelor and post-bachelor May 2005 graduates had completed at least 
one developmental education course. The number rose to 21 percent 
when the master degree candidates were removed. In fact, not only did 
these students survive, they thrived. For the May 2005 commencement 
exercises, 57 percent of students nominated for student awards had 
enrolled in at least one developmental education course during their 
academic careers; furthermore, 26 percent of the students who earned 
individual awards had enrolled in at least one developmental education 
course. The 2005-2006 vice president of the student senate began his 
academic career as a developmental education student, and the 2006 
student senator for the College of Technology previously completed 
developmental education math and English courses. The data showed 
that developmental education students survive, thrive and benefit the 
university community when provided with the necessary foundation. 

Conclusion 

Flaving the data builds credibility. Using the data builds a better 
program and improves student success. The high ranking achieved by 
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the Academic Support Center on a recent campus-wide Noel-Levitz 
student satisfaction survey highlighted the program’s success. During a 
recent accreditation committee meeting, the committee chair noted the 
quality of the quantitative assessment conducted by the developmental 
education department and urged other departments to follow suit. 
Carefully gathering, analyzing and sharing data paves a program’s path 
through the somewhat dense and often tangled developmental education 
forest of identifying and serving diverse student needs. As we say, “If the 
numbers are right, you can’t go wrong.” 


References 

Boylan, H. R., Bliss, L. B., & Bonham, B. S. (1997). Program components 
and their relationship to student performance. Journal of Developmental 
Education, 20(3), 2-8. 

Boylan, H. R. & Saxon, D. P. (n.d.). Outcomes of Remediation. Paper prepared 
for The League of Innovation in the Community College. Retrieved 
February 8, 2005, from http://www.ncde.appstate.edu/reserve_reading/ 
O utcomes_of_ Remediation. htm 

Carroll, L. (1957). Alice in wonderland. Philadelphia: The John C. Winston 
Company. 

Hendriksen, S. I., Yang, L., Love, B. & Hall, M. C. (Spring 2005). Assessing 
academic support: The effects of tutoring on student learning outcomes. 
Journal of College Reading and Learning, 35(2), 56-65. 

Simpson, M. L. (Winter 2002). Program evaluation studies: Learning delivery 
model suggestions . Journal of Developmental Education, 26(2), 2-10. 


Leanne H. Frost received concurrent Master’s of Education degrees in Reading and 
Interdisciplinary Studies from MSU-Billings in 2005■ Her thesis examined the 
relationship between reading ability and student success at the postsecondary level 
and provided the impetus for the development of a reading improvement course. She 
currently teaches writing and reading at MSU-Billings College of Technology and 
serves as Assistant Director of the Academic Support Center. 

Gwendolyn K. Brauns math teaching experience spans all levels, from elementary 
to postsecondary. As Director of the Academic Support Center and Developmental 
Education at MSU-Billings, she has been instrumental in uniting the developmental 
courses. Her degrees include a BS in Elementary Education and Secondary 
Mathematics from Lincoln University and a Master’s ofEducation in Educational 
Technology from MSU-Billings. 





